Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support ESLint v9 #2996

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
Draft

Conversation

G-Rath
Copy link
Contributor

@G-Rath G-Rath commented Apr 7, 2024

Resolves #2948

Copy link

codecov bot commented Apr 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.14%. Comparing base (a9018a8) to head (eba7cb0).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2996      +/-   ##
==========================================
- Coverage   95.31%   95.14%   -0.18%     
==========================================
  Files          82       82              
  Lines        3438     3439       +1     
  Branches     1186     1188       +2     
==========================================
- Hits         3277     3272       -5     
- Misses        161      167       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@G-Rath G-Rath force-pushed the support-eslint-v9 branch 2 times, most recently from 7c59670 to 543e378 Compare April 7, 2024 01:10
@G-Rath

This comment was marked as outdated.

@ljharb
Copy link
Member

ljharb commented Apr 8, 2024

I've just gotten back from travel, so I'll try to take a look at it in the coming days.

One likely obstacle is that all the tests assume eslintrc, but eslint 9 requires an env var to support it, otherwise it defaults to flat config. The initial support needs to test eslintrc, and it would be fine to add flat config tests in a follow-on if needed.

It's likely that the way eslint < 9's RuleTester works is subtly different than in eslint 9, so we may need an abstraction to handle that.

@G-Rath
Copy link
Contributor Author

G-Rath commented Apr 8, 2024

Yup part of this includes switching to a custom rule tester that converts the tests from eslintrc to flat config if they're running on v9, which is what we've been using in eslint-plugin-jest without issue.

@ljharb
Copy link
Member

ljharb commented Apr 8, 2024

That's great, but testing eslintrc on eslint 9 is also very important.

It'd also be great if there was a commit or two I could land separately, that worked on eslint < 9, so that only the 9-specific parts remained in this PR after that was landed and rebased :-)

@G-Rath
Copy link
Contributor Author

G-Rath commented Apr 8, 2024

If you're meaning the context helper stuff, yeah I'm happy to pull them out I just figured you'd have asked them to be tested against ESLint v9 to prove they actually worked :-)

@ljharb
Copy link
Member

ljharb commented Apr 8, 2024

You figured right :-) but i can just cherry-pick the commits out of this PR once things are working.

@G-Rath
Copy link
Contributor Author

G-Rath commented Apr 8, 2024

Right well they're already good for cherry-picking - you want to pick from e31fe5c...543e378 (sans f0853cb once #2997 is landed), and away you go.

I think you can have the eslintrc-on-v9 support by just running the whole test suite twice with the env enabled and an extra test in the custom rule tester - I'll push that up shortly

@G-Rath
Copy link
Contributor Author

G-Rath commented Apr 8, 2024

@ljharb RuleTester does not support eslintrc in v9 - the env variable is only used by the CLI

@ljharb
Copy link
Member

ljharb commented Apr 8, 2024

well that sucks, we’ll need to file an eslint issue about that gap.

@G-Rath
Copy link
Contributor Author

G-Rath commented Apr 8, 2024

I'll leave that to you as I suspect you'll have more pull :-)

@ljharb
Copy link
Member

ljharb commented Apr 8, 2024

Filed eslint/eslint#18292.

tests/src/rule-tester.js Outdated Show resolved Hide resolved
Copy link
Member

@ljharb ljharb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help me understand what withoutAutofixOutput is for?

const scopeBody = getScopeBody(getScope(context, node));
log('got scope:', scopeBody);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the scope is for the program, i think, so it should be gotten without the node, or with the entire program's node?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet, I'll try passing in the program node and see what happens

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this still needs addressing

@@ -32,7 +32,7 @@ ruleTester.run('no-cycle', rule, {
test({ code: 'var foo = require("./")' }),
test({ code: 'var foo = require("@scope/foo")' }),
test({ code: 'var bar = require("./bar/index")' }),
test({ code: 'var bar = require("./bar")' }),
...usingFlatConfig ? [] : [test({ code: 'var bar = require("./bar")' })],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can't this one work in flat config?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it's considered a duplicate of the next line and v9 ruletester doesn't allow duplicate tests

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not, though - it has a different filename. v9 doesn't consider that to be different? sounds like a bug in v9.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, feel free to raise an issue over at eslint

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in that case, let's add a code comment or something so we can keep both test cases.

@G-Rath

This comment was marked as outdated.

@ljharb
Copy link
Member

ljharb commented Apr 12, 2024

Totally, while it's still a draft I'm indeed just chipping away at the review :-) your effort is appreciated!

src/context.js Outdated Show resolved Hide resolved
@G-Rath
Copy link
Contributor Author

G-Rath commented Sep 6, 2024

@ljharb I think I've done that split right, but let me know if not 😅 (306bbb8 & fd0d97f)

@G-Rath
Copy link
Contributor Author

G-Rath commented Sep 8, 2024

Alright, so I think I've tracked this down to visitorKeys being null which is passed to visit which is required by visit.

That seems to happen because eslint-plugin-import handles resolving that based on the parserPath when a parser does not itself provide that, but parserPath is not provided in flat config 😬 (instead, we have the actual parser itself, though it looks like it might be wrapped anyway?)

This means it's an existing bug in the flat config handling, which I can produce using v8.

@ljharb would you be able to look into this and suggest some solutions? I'm not sure the best way to deal with it especially in a manner that preserves full backwards compatibility.

@michaelfaith
Copy link
Contributor

michaelfaith commented Sep 9, 2024

Alright, so I think I've tracked this down to visitorKeys being null which is passed to visit which is required by visit.

Which build issue is this related to? The cases where the rule isn't reporting an error when it should, or these parseForEslint parsing errors in the logs?

Error while parsing /home/runner/work/eslint-plugin-import/eslint-plugin-import/tests/files/no-unused-modules/arbitrary-module-namespace-identifier-name-b.js
Line 1, column 9: Identifier expected.
`parseForESLint` from parser ``context.languageOptions.parser`` is invalid and will just be ignored

And is there possibly a correlation between those parsing errors and where you're seeing visitorKeys as null? I'm guessing so, since the fallback call to keysFromParser passes in undefined for parsedResult, and both conditions in keysFromParser when parsedResult isn't provided, rely on parserPath being there.

@G-Rath
Copy link
Contributor Author

G-Rath commented Sep 9, 2024

Which build issue is this related to?

I'm working on no-cycle (though I now know this will def fix some of the other failures)

And is there possibly a correlation between those parsing errors and where you're seeing visitorKeys as null?

No, no-cycle does not generate those warnings/errors - so far the only rule I can find that generates those is no-unused-modules.

They're also happening on main e.g. https://github.com/import-js/eslint-plugin-import/actions/runs/10745866324/job/29830827603

(it's because if the parser errors, then parse continues after printing a warning leading to ast being undefined which then triggers this warning - and the parser is erroring because it's being told to parse invalid syntax, so I'm pretty sure this is expected)

I'm guessing so, since the fallback call to keysFromParser passes in undefined for parsedResult, and both conditions in keysFromParser when parsedResult isn't provided, rely on parserPath being there.

Either I'm not following you or I don't think that's correct - from what I can see, technically you're right in saying "fallback" but the comments within keysFromParser say that parsedResult.visitorKeys is expected only from @typescript-eslint/parser and @babel/eslint-parser, otherwise parserPath is required and that includes for the default espree parser.

@michaelfaith
Copy link
Contributor

michaelfaith commented Sep 10, 2024

No, no-cycle does not generate those warnings/errors - so far the only rule I can find that generates those is no-unused-modules.

That's actually not true. This one is from no-cycle, and the others are across a variety of tests, not just no-unused-modules.

image

image

They're also happening on main e.g. https://github.com/import-js/eslint-plugin-import/actions/runs/10745866324/job/29830827603

They're happening on main in v8 and earlier, which have parserPath to fall back on when they hit a parsing error... I think it's worth exploring, but I can grab your branch and help investigate.

@G-Rath
Copy link
Contributor Author

G-Rath commented Sep 10, 2024

That's actually not true. This one is from no-cycle, and the others are across a variety of tests, not just no-unused-modules.

That's actually not true:

eslint-plugin-import on  support-eslint-v9 [!] is 📦 v2.30.0 via  v20.11.0 took 13s
❯ npm run mocha tests/src/rules/no-cycle.js

> eslint-plugin-import@2.30.0 mocha
> cross-env BABEL_ENV=test nyc mocha tests/src/rules/no-cycle.js



  ․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․․
  ․․․․․․․․․․!!․․!!․․․․․․

  134 passing (723ms)
  4 failing

  1) no-cycle invalid import { foo } from "./es6/depth-one-dynamic"; // #2265 6:

      AssertionError [ERR_ASSERTION] [ERR_ASSERTION]: Should have 1 error but had 0: []
      + expected - actual

      -0
      +1

      at testInvalidTemplate (node_modules/eslint/lib/rule-tester/rule-tester.js:998:24)
      at Context.<anonymous> (node_modules/eslint/lib/rule-tester/rule-tester.js:1274:33)
      at processImmediate (node:internal/timers:478:21)

  2) no-cycle invalid import { foo } from "./es6/depth-one-dynamic"; // #2265 6 // disableScc=true:

      AssertionError [ERR_ASSERTION] [ERR_ASSERTION]: Should have 1 error but had 0: []
      + expected - actual

      -0
      +1

      at testInvalidTemplate (node_modules/eslint/lib/rule-tester/rule-tester.js:998:24)
      at Context.<anonymous> (node_modules/eslint/lib/rule-tester/rule-tester.js:1274:33)
      at processImmediate (node:internal/timers:478:21)

  3) no-cycle invalid import { foo } from "./es6/depth-one-dynamic"; // #2265 8:

      AssertionError [ERR_ASSERTION] [ERR_ASSERTION]: Should have 1 error but had 0: []
      + expected - actual

      -0
      +1

      at testInvalidTemplate (node_modules/eslint/lib/rule-tester/rule-tester.js:998:24)
      at Context.<anonymous> (node_modules/eslint/lib/rule-tester/rule-tester.js:1274:33)
      at processImmediate (node:internal/timers:478:21)

  4) no-cycle invalid import { foo } from "./es6/depth-one-dynamic"; // #2265 8 // disableScc=true:

      AssertionError [ERR_ASSERTION] [ERR_ASSERTION]: Should have 1 error but had 0: []
      + expected - actual

      -0
      +1

      at testInvalidTemplate (node_modules/eslint/lib/rule-tester/rule-tester.js:998:24)
      at Context.<anonymous> (node_modules/eslint/lib/rule-tester/rule-tester.js:1274:33)
      at processImmediate (node:internal/timers:478:21)

The error in your screenshot is coming from no-unused-modules:

image

Also, I'm guessing you missed when I said that I could reproduce this using ESLint v8 on a simple reproduction, which also doesn't generate that parser error:

workspace/projects-scrap/eslint-import is 📦 v1.0.0 via  v20.11.0
❯ tree -I node_modules
.
├── a.js
├── depth-zero.js
├── es6
│   └── depth-one-dynamic.js
├── eslint.config.js
├── eslint.config.old.js
├── isv
│   ├── package-lock.json
│   └── package.json
├── named-exports.js
├── package-lock.json
├── package.json
├── rule.js
├── script.js
└── test.js

2 directories, 13 files

workspace/projects-scrap/eslint-import is 📦 v1.0.0 via  v20.11.0
❯ ESLINT_USE_FLAT_CONFIG=false npx eslint@8 --config eslint.config.old.js .

/home/jones/workspace/projects-scrap/eslint-import/depth-zero.js
  3:1  error  Dependency cycle detected  import/no-cycle

/home/jones/workspace/projects-scrap/eslint-import/es6/depth-one-dynamic.js
  1:26  error  Dependency cycle detected  import/no-cycle

✖ 2 problems (2 errors, 0 warnings)


workspace/projects-scrap/eslint-import is 📦 v1.0.0 via  v20.11.0
❯ ESLINT_USE_FLAT_CONFIG=true npx eslint@8 --config eslint.config.js .

I'm not saying it should be completely ignored, but from what I can tell it isn't the issue here - I do agree it's interesting that it doesn't seem to be happening everywhere those parsers are being used, but it does seems like its happening for dynamic import tests only and maybe that's only where this code path matters.

@michaelfaith
Copy link
Contributor

michaelfaith commented Sep 11, 2024

So, I have a solution for addressing the visitorKeys issue for the espree use-case. However, it may be harder to get the "Old" Babel use case to be possible with flat-config / v9+, since it entirely depends on digging into the package file structure within node_modules and trying to resolve its visitor-keys.js module.

function getBabelEslintVisitorKeys(parserPath) {
  if (parserPath.endsWith('index.js')) {
    const hypotheticalLocation = parserPath.replace('index.js', 'visitor-keys.js');
    if (fs.existsSync(hypotheticalLocation)) {
      const keys = moduleRequire(hypotheticalLocation);
      return keys.default || keys;
    }
  }
  return null;
}

I guess we could try and use require.resolve here to find where babel-eslint is, in place of parserPath, but wanted to check before going down that road if we need to support the old babel parser with flat config? I haven't checked to see what a newer babel-eslint would look like, since it may have an easier api to get ahold of, but there's explicit tests around using this old version of babel, so that didn't seem like the right path either. Thoughts?

And @G-Rath I didn't miss anything you said, it's just that correlation isn't causation, and I'm trying to focus on identifying root causes.

@ljharb
Copy link
Member

ljharb commented Sep 11, 2024

hmm, i guess that's a fair point that we don't actually have to support it with flat config, as long as we continue doing so for eslintrc. I'm fine with that as long as the error message is clear, and that tests are covering the behavior.

@michaelfaith
Copy link
Contributor

michaelfaith commented Sep 12, 2024

Ok, I believe I've figured out the issue with no-cycle. After implementing the fix for the espree parser (though that may not actually be involved), what I was left with for errors in the no-cycle test were two tests failing (twice, once with disableScc and once without it). Both tests were in the invalid group reporting no errors (so failing the test). And both of them were related to this issue from a few years ago related to dynamic imports: #2265

testVersion('> 3', () => ({ // Dynamic import is not properly caracterized with eslint < 4
  code: `import { foo } from "./${testDialect}/depth-one-dynamic"; // #2265 6`,
  errors: [error(`Dependency cycle detected.`)],
  parser: parsers.BABEL_OLD,
})),
test({
  code: `import { foo } from "./${testDialect}/depth-one-dynamic"; // #2265 8`,
  errors: [error(`Dependency cycle detected.`)],
  parser: parsers.TS_NEW,
}),

After some exploration, I found that the tests actually passed (correctly reported the error), if I didn't run the valid versions of the test which turns on allowUnsafeDynamicCyclicDependency! If i ran either of the dynamic import tests with allowUnsafeDynamicCyclicDependency turned on, then the tests that should have reported failure didn't. So that led me to believe we were dealing with a caching issue. And sure enough, if I turned the cache off for the rule, it correctly reported the violation.

test({
  code: `import { foo } from "./${testDialect}/depth-one-dynamic"; // #2265 8`,
  errors: [error(`Dependency cycle detected.`)],
  parser: parsers.TS_NEW,
  settings: {
    'import/cache': {
      lifetime: 0,
    },
  }
}),

It's just a theory based on these explorations, but I think it could be pointing towards a root cause. Will explore more as I'm able.

@ljharb
Copy link
Member

ljharb commented Sep 12, 2024

That's great to hear - a separate PR to fix the caching bug would be really helpful (and it's fine if tests aren't practical to write for that)

@michaelfaith
Copy link
Contributor

michaelfaith commented Sep 12, 2024

a separate PR to fix the caching bug would be really helpful

What I haven't had time to check, though, is if this caching issue is being caused by how this PR wrap the v9 RuleTester. It could be that that's an issue introduced here, rather than it being a bug with the cache system itself. That's what I want to look at next. If it is an existing bug, then yeah, I'll do a separate PR for that (and for the parser issue)

@michaelfaith
Copy link
Contributor

What I haven't had time to check, though, is if this caching issue is being caused by how this PR wrap the v9 RuleTester.

Quick update on this. It happens without the wrapped RuleTester too, so I don't think that's affecting what I was seeing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Support eslint v9